Hyperparameter Tuning

ML models - Functions with parameters and hyper-parameters

• Parameters - Change during training
• Hyper-parameters - Set before training

Sensitive to batch size and learning rate (dataset-dependent)

Batch size - Control number of samples (calculated gradient)

• Too small - Bounce around
• Too large - Take long time

Learning rate - Control size of step (weight space)

• Too small - Take long time
• Too large - Bounce around